Skip to content

Support numpy 2.0.0b1 - #432

Merged
stefanseefeld merged 1 commit into
boostorg:developfrom
duburcqa:patch-1
May 6, 2024
Merged

Support numpy 2.0.0b1#432
stefanseefeld merged 1 commit into
boostorg:developfrom
duburcqa:patch-1

Conversation

@duburcqa

Copy link
Copy Markdown
Contributor

No description provided.

@duburcqa

Copy link
Copy Markdown
Contributor Author

Can someone review this MR ? numpy==2.0 is going to be release soon and it would be nice to merge support of it before it happens.

@mattip

mattip commented May 6, 2024

Copy link
Copy Markdown

Fixes #431

@stefanseefeld

Copy link
Copy Markdown
Member

many thanks !

JArmandoAnaya added a commit to JArmandoAnaya/carla that referenced this pull request Apr 14, 2026
Apply the fix from boostorg/python#432 to Boost 1.84.0 via a build-time
patch. Adds PyDataType_* accessor macro shims for NumPy 1.x so the same
call sites in dtype.cpp compile on both NumPy 1.x and 2.x ABIs (NEP 52).

- Add Util/Patches/boost-1.84.0/dtype.cpp.patch
- Add Util/Patches/boost-1.84.0/verify_patch.{cpp,sh} compile contract test
- Apply patch in Setup.sh after Boost extraction
- Widen requirements from numpy<2.0.0 to numpy>=1.18.4 (3 files)
- Update Docker test requirements: 3.11 and 3.12 to numpy>=2.0.0 for CI
  NumPy 2 coverage, 3.10 remains on numpy==1.24.4 for NumPy 1 regression guard
- Add test_numpy_compat.py (15 tests across 5 classes)
- Add run_numpy_compat_matrix.sh for automated NumPy 1/2 local validation
JArmandoAnaya added a commit to JArmandoAnaya/carla that referenced this pull request Apr 14, 2026
Apply the fix from boostorg/python#432 to Boost 1.84.0 via a build-time
patch. Adds PyDataType_* accessor macro shims for NumPy 1.x so the same
call sites in dtype.cpp compile on both NumPy 1.x and 2.x ABIs (NEP 52).

- Add Util/Patches/boost-1.84.0/dtype.cpp.patch
- Add Util/Patches/boost-1.84.0/verify_patch.{cpp,sh} compile contract test
- Apply patch in Setup.sh after Boost extraction
- Widen requirements from numpy<2.0.0 to numpy>=1.18.4 (3 files)
- Update Docker test requirements: 3.11 and 3.12 to numpy>=2.0.0 for CI
  NumPy 2 coverage, 3.10 remains on numpy==1.24.4 for NumPy 1 regression guard
- Add test_numpy_compat.py (15 tests across 5 classes)
- Add run_numpy_compat_matrix.sh for automated NumPy 1/2 local validation
JArmandoAnaya added a commit to JArmandoAnaya/carla that referenced this pull request Apr 14, 2026
Apply the fix from boostorg/python#432 to Boost 1.84.0 via a build-time
patch. Adds PyDataType_* accessor macro shims for NumPy 1.x so the same
call sites in dtype.cpp compile on both NumPy 1.x and 2.x ABIs (NEP 52).

- Add Util/Patches/boost-1.84.0/dtype.cpp.patch
- Add Util/Patches/boost-1.84.0/verify_patch.{cpp,sh} compile contract test
- Apply patch in Setup.sh after Boost extraction
- Widen requirements from numpy<2.0.0 to numpy>=1.18.4 (3 files)
- Update Docker test requirements: 3.11 and 3.12 to numpy>=2.0.0 for CI
  NumPy 2 coverage, 3.10 remains on numpy==1.24.4 for NumPy 1 regression guard
- Add test_numpy_compat.py (15 tests across 5 classes)
- Add run_numpy_compat_matrix.sh for automated NumPy 1/2 local validation
JArmandoAnaya added a commit to JArmandoAnaya/carla that referenced this pull request Apr 14, 2026
Apply the fix from boostorg/python#432 to Boost 1.84.0 via a build-time
patch. Adds PyDataType_* accessor macro shims for NumPy 1.x so the same
call sites in dtype.cpp compile on both NumPy 1.x and 2.x ABIs (NEP 52).

- Add Util/Patches/boost-1.84.0/dtype.cpp.patch
- Add Util/Patches/boost-1.84.0/verify_patch.{cpp,sh} compile contract test
- Apply patch in Setup.sh after Boost extraction
- Widen requirements from numpy<2.0.0 to numpy>=1.18.4 (3 files)
- Update Docker test requirements: 3.11 and 3.12 to numpy>=2.0.0 for CI
  NumPy 2 coverage, 3.10 remains on numpy==1.24.4 for NumPy 1 regression guard
- Add test_numpy_compat.py (15 tests across 5 classes)
- Add run_numpy_compat_matrix.sh for automated NumPy 1/2 local validation
JArmandoAnaya added a commit to JArmandoAnaya/carla that referenced this pull request Apr 14, 2026
Drop the in-tree dtype.cpp.patch against Boost 1.84.0 and upgrade to
Boost 1.89.0, which carries the upstream NumPy 2 C ABI fix from
boostorg/python#432 natively. libboost_numpy now compiles against both
NumPy 1.x (>=1.18.4) and NumPy 2.x headers without any vendored patches.

Boost 1.84 -> 1.89 removed several deprecated Asio and Filesystem APIs;
all call sites in LibCarla and the CarlaTools UE4 plugin updated:
- io_context::work -> executor_work_guard (removed 1.85)
- io_context::reset() -> restart() (removed 1.86)
- address::from_string() -> make_address() (removed 1.85)
- resolver::query/iterator -> range-based resolve() (removed 1.85)
- buffer_cast<T>() -> static_cast via streambuf::data().data() (removed 1.88)
- io_service typedef -> io_context (removed 1.85)
- boost/filesystem/operations.hpp no longer includes directory.hpp (1.85)

Windows installer (install_boost.bat) SHA256 updated to the 1.89.0 zip
digest. MSVC C4459 pragma guard added around boost/asio.hpp in
MapPreviewUserWidget.h to prevent warnings-as-errors failure.

New regression-guard tests: test_boost_deadline_timer.cpp (3 tests),
test_boost_rtree.cpp (4 tests), test_boost_version.py.
JArmandoAnaya added a commit to JArmandoAnaya/carla that referenced this pull request Apr 16, 2026
Drop the in-tree dtype.cpp.patch against Boost 1.84.0 and upgrade to
Boost 1.89.0, which carries the upstream NumPy 2 C ABI fix from
boostorg/python#432 natively. libboost_numpy now compiles against both
NumPy 1.x (>=1.18.4) and NumPy 2.x headers without any vendored patches.

Boost 1.84 -> 1.89 removed several deprecated Asio and Filesystem APIs;
all call sites in LibCarla and the CarlaTools UE4 plugin updated:
- io_context::work -> executor_work_guard (removed 1.85)
- io_context::reset() -> restart() (removed 1.86)
- address::from_string() -> make_address() (removed 1.85)
- resolver::query/iterator -> range-based resolve() (removed 1.85)
- buffer_cast<T>() -> static_cast via streambuf::data().data() (removed 1.88)
- io_service typedef -> io_context (removed 1.85)
- boost/filesystem/operations.hpp no longer includes directory.hpp (1.85)

Windows installer (install_boost.bat) SHA256 updated to the 1.89.0 zip
digest. MSVC C4459 pragma guard added around boost/asio.hpp in
MapPreviewUserWidget.h to prevent warnings-as-errors failure.

New regression-guard tests: test_boost_deadline_timer.cpp (3 tests),
test_boost_rtree.cpp (4 tests), test_boost_version.py.
LuisPovedaCano pushed a commit to carla-simulator/carla that referenced this pull request Apr 17, 2026
#9670)

* fix(PythonAPI/examples): replace deprecated NumPy aliases for NumPy 2 compatibility

Replace np.bool with np.bool_ in DVS event dtype in V2XDemo.py,
manual_control_chrono.py, and manual_control_carsim.py (removed in
NumPy 1.24). Replace np.matrix with np.array in client_bounding_boxes.py
(deprecated in NumPy 1.x, removed in NumPy 2.x). All replacements are
backward-compatible with NumPy >= 1.18.4.

* fix(build): upgrade Boost to 1.89.0 for NumPy 2 ABI compatibility

Drop the in-tree dtype.cpp.patch against Boost 1.84.0 and upgrade to
Boost 1.89.0, which carries the upstream NumPy 2 C ABI fix from
boostorg/python#432 natively. libboost_numpy now compiles against both
NumPy 1.x (>=1.18.4) and NumPy 2.x headers without any vendored patches.

Boost 1.84 -> 1.89 removed several deprecated Asio and Filesystem APIs;
all call sites in LibCarla and the CarlaTools UE4 plugin updated:
- io_context::work -> executor_work_guard (removed 1.85)
- io_context::reset() -> restart() (removed 1.86)
- address::from_string() -> make_address() (removed 1.85)
- resolver::query/iterator -> range-based resolve() (removed 1.85)
- buffer_cast<T>() -> static_cast via streambuf::data().data() (removed 1.88)
- io_service typedef -> io_context (removed 1.85)
- boost/filesystem/operations.hpp no longer includes directory.hpp (1.85)

Windows installer (install_boost.bat) SHA256 updated to the 1.89.0 zip
digest. MSVC C4459 pragma guard added around boost/asio.hpp in
MapPreviewUserWidget.h to prevent warnings-as-errors failure.

New regression-guard tests: test_boost_deadline_timer.cpp (3 tests),
test_boost_rtree.cpp (4 tests), test_boost_version.py.

* docs(CHANGELOG): note NumPy 2 compatibility

* fix(build): bump Boost 1.89 -> 1.90 and migrate timers to steady_timer

Boost 1.90 added [[deprecated]] annotations to boost::asio::deadline_timer.
LibCarla's networking code is migrated to boost::asio::steady_timer across
Primary, Secondary, Client, and ServerSession.

steady_timer default-constructs with expiry() == time_point::min() (past),
whereas deadline_timer defaulted to pos_infin (far future). The recursive
StartTimer() guard in ServerSession::StartTimer and Primary::StartTimer
checks expiry() <= now() and would fire Close() before socket setup if left
at the new default, so both _deadline members are explicitly constructed
with time_point::max() to preserve the old semantics.

API migrations at the call sites:
  * expires_from_now(duration)          -> expires_after(duration)
  * expires_at()                        -> expiry()
  * deadline_timer::traits_type::now()  -> std::chrono::steady_clock::now()
  * boost::posix_time durations         -> std::chrono durations

Build scripts:
  * Util/BuildTools/Setup.sh:      BOOST_VERSION 1.89.0 -> 1.90.0, SHA256
  * Util/BuildTools/Setup.bat:     version bump (Windows)
  * Util/BuildTools/BuildOSMRenderer.bat: install path bump (Windows)
  * Util/InstallersWin/install_boost.bat: source-zip SHA256 bump

Tests:
  * test_boost_deadline_timer.cpp: test class BoostDeadlineTimer renamed
    to BoostAsioTimer and migrated to steady_timer; 3 new regression
    tests pin the contract that prevents future refactors from silently
    reintroducing the default-construction trap:
      - InitializedWithMaxIsInDistantFuture
      - DefaultConstructedExpiryIsInPast
      - CancelTriggersAbortedError
  * test_boost_rtree.cpp:          header comment 1.89 -> 1.90
  * test_boost_version.py:         EXPECTED_BOOST_VERSION 1.89.0 -> 1.90.0;
                                   stale-cache check rolled 1.84.0 -> 1.89.0
  * CHANGELOG.md:                  NumPy 2 entry references Boost 1.90.0

Verified: make LibCarla + full make check.LibCarla pass on server and
client, release and debug. All 6 BoostAsioTimer.* and 4 BoostRtree.* tests
pass on both suites.

* fix(build): guard against all pre-1.90 Boost installs

Address review on PR #9670. The previous test guarded only against the 1.89.0 intermediate install used during development, letting a direct
  1.84.0 -> 1.90.0 upgrade slips past undetected.

- Replace the negative-only LDD check with a positive assertion that libboost_*.so.1.90.0 is present in PythonAPI/carla/dependencies/lib/, plus an explicit guard over known stale baselines (1.84.0, 1.89.0). Boost is statically linked into libcarla.so, so ldd cannot observe it; the dependencies/lib symlinks are the authoritative record.
- Update two Linux build scripts still pinned to boost-1.84.0 that were missed in the 1.84->1.89->1.90 bumps (the Windows .bat was updated):
  - Util/BuildTools/BuildOSMRenderer.sh
  - Util/BuildTools/Ad-rss.sh
LuisPovedaCano pushed a commit to carla-simulator/carla that referenced this pull request May 21, 2026
…9737)

* fix(PythonAPI/build): port NumPy 2 compatibility via Boost 1.90 upgrade

Upgrades Boost from 1.84.0 to 1.90.0 to pick up the upstream NumPy 2 C
ABI fix (boostorg/python#432). LibCarla networking migrated from
boost::asio::deadline_timer to boost::asio::steady_timer, and the
deprecated io_service, address::from_string, resolver::query,
buffer_cast, and io_context::work APIs replaced across LibCarla and the
CarlaTools UE5 plugin. The CMake-level NumPy 2 guard in
PythonAPI/CMakeLists.txt is removed since Boost.Python.Numpy supports
NumPy 2 from 1.89+.

steady_timer default-constructs with expiry() == time_point::min() (in
the past), whereas deadline_timer defaulted to pos_infin.
Primary::_deadline and ServerSession::_deadline are explicitly
initialised with std::chrono::steady_clock::time_point::max() so the
recursive StartTimer() guard does not fire Close() before socket setup.

ue5-dev uses CMake FetchContent rather than the ue4-dev Setup.sh, so the
version pin lives in CMake/Options.cmake (CARLA_BOOST_VERSION). Boost
1.90 also changed the GitHub release asset name, so
CMake/Dependencies.cmake now fetches the boost-1.90.0-cmake.zip variant.

NumPy 1.x (>= 1.24.4) support is preserved, not dropped. Requirements
files relax numpy<2.0,>=1.24.4 to numpy>=1.24.4.

Regression guards: 6 GTest cases in
LibCarla/source/test/common/test_boost_asio_timer.cpp pin the
steady_timer contract including the time_point::max() construction
trap, 4 GTest cases in test_boost_rtree.cpp guard Boost.Geometry rtree
behaviour, and two Python unit tests under PythonAPI/test/unit/ pin the
version source of truth and the prop_control.py np.bool_ migration.

(adapted from ue4-dev 6e15235)

* fix(PythonAPI/build): address Copilot review on PR #9737

- Read streambufs via boost::asio::buffers_begin/end in
  MapPreviewUserWidget instead of relying on
  streambuf::data().data().
- Rename CarlaTools plugin member io_service to io_context to
  match the migrated type.
- Add <algorithm> include in test_boost_rtree.cpp for std::sort.
- Narrow except ImportError to except ModuleNotFoundError in
  test_boost_version.py so real load failures surface.
donghaiwang added a commit to OpenHUTB/hutb that referenced this pull request Aug 19, 2026
…0 upgrade (#9670)

* fix(PythonAPI/examples): replace deprecated NumPy aliases for NumPy 2 compatibility

Replace np.bool with np.bool_ in DVS event dtype in V2XDemo.py,
manual_control_chrono.py, and manual_control_carsim.py (removed in
NumPy 1.24). Replace np.matrix with np.array in client_bounding_boxes.py
(deprecated in NumPy 1.x, removed in NumPy 2.x). All replacements are
backward-compatible with NumPy >= 1.18.4.

* fix(build): upgrade Boost to 1.89.0 for NumPy 2 ABI compatibility

Drop the in-tree dtype.cpp.patch against Boost 1.84.0 and upgrade to
Boost 1.89.0, which carries the upstream NumPy 2 C ABI fix from
boostorg/python#432 natively. libboost_numpy now compiles against both
NumPy 1.x (>=1.18.4) and NumPy 2.x headers without any vendored patches.

Boost 1.84 -> 1.89 removed several deprecated Asio and Filesystem APIs;
all call sites in LibCarla and the CarlaTools UE4 plugin updated:
- io_context::work -> executor_work_guard (removed 1.85)
- io_context::reset() -> restart() (removed 1.86)
- address::from_string() -> make_address() (removed 1.85)
- resolver::query/iterator -> range-based resolve() (removed 1.85)
- buffer_cast<T>() -> static_cast via streambuf::data().data() (removed 1.88)
- io_service typedef -> io_context (removed 1.85)
- boost/filesystem/operations.hpp no longer includes directory.hpp (1.85)

Windows installer (install_boost.bat) SHA256 updated to the 1.89.0 zip
digest. MSVC C4459 pragma guard added around boost/asio.hpp in
MapPreviewUserWidget.h to prevent warnings-as-errors failure.

New regression-guard tests: test_boost_deadline_timer.cpp (3 tests),
test_boost_rtree.cpp (4 tests), test_boost_version.py.

* docs(CHANGELOG): note NumPy 2 compatibility

* fix(build): bump Boost 1.89 -> 1.90 and migrate timers to steady_timer

Boost 1.90 added [[deprecated]] annotations to boost::asio::deadline_timer.
LibCarla's networking code is migrated to boost::asio::steady_timer across
Primary, Secondary, Client, and ServerSession.

steady_timer default-constructs with expiry() == time_point::min() (past),
whereas deadline_timer defaulted to pos_infin (far future). The recursive
StartTimer() guard in ServerSession::StartTimer and Primary::StartTimer
checks expiry() <= now() and would fire Close() before socket setup if left
at the new default, so both _deadline members are explicitly constructed
with time_point::max() to preserve the old semantics.

API migrations at the call sites:
  * expires_from_now(duration)          -> expires_after(duration)
  * expires_at()                        -> expiry()
  * deadline_timer::traits_type::now()  -> std::chrono::steady_clock::now()
  * boost::posix_time durations         -> std::chrono durations

Build scripts:
  * Util/BuildTools/Setup.sh:      BOOST_VERSION 1.89.0 -> 1.90.0, SHA256
  * Util/BuildTools/Setup.bat:     version bump (Windows)
  * Util/BuildTools/BuildOSMRenderer.bat: install path bump (Windows)
  * Util/InstallersWin/install_boost.bat: source-zip SHA256 bump

Tests:
  * test_boost_deadline_timer.cpp: test class BoostDeadlineTimer renamed
    to BoostAsioTimer and migrated to steady_timer; 3 new regression
    tests pin the contract that prevents future refactors from silently
    reintroducing the default-construction trap:
      - InitializedWithMaxIsInDistantFuture
      - DefaultConstructedExpiryIsInPast
      - CancelTriggersAbortedError
  * test_boost_rtree.cpp:          header comment 1.89 -> 1.90
  * test_boost_version.py:         EXPECTED_BOOST_VERSION 1.89.0 -> 1.90.0;
                                   stale-cache check rolled 1.84.0 -> 1.89.0
  * CHANGELOG.md:                  NumPy 2 entry references Boost 1.90.0

Verified: make LibCarla + full make check.LibCarla pass on server and
client, release and debug. All 6 BoostAsioTimer.* and 4 BoostRtree.* tests
pass on both suites.

* fix(build): guard against all pre-1.90 Boost installs

Address review on PR #9670. The previous test guarded only against the 1.89.0 intermediate install used during development, letting a direct
  1.84.0 -> 1.90.0 upgrade slips past undetected.

- Replace the negative-only LDD check with a positive assertion that libboost_*.so.1.90.0 is present in PythonAPI/carla/dependencies/lib/, plus an explicit guard over known stale baselines (1.84.0, 1.89.0). Boost is statically linked into libcarla.so, so ldd cannot observe it; the dependencies/lib symlinks are the authoritative record.
- Update two Linux build scripts still pinned to boost-1.84.0 that were missed in the 1.84->1.89->1.90 bumps (the Windows .bat was updated):
  - Util/BuildTools/BuildOSMRenderer.sh
  - Util/BuildTools/Ad-rss.sh
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants